Fix streaming support for multi property configurations #257
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…operty projects" This reverts commit c20d5b1.
dgitis
approved these changes
Jul 31, 2023
There was a problem hiding this comment.
My concern with this PR is that the qualify statement at the end won't deduplicate in all situations. I am not aware of any event_params that get added when Google processes the GA4 data, but just one field would cause the deduplication to stop working.
I'm also concerned that deduplicating this way will take a lot of unnecessary processing, but I can't think of a better way of doing this.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description & motivation
Resolves #254
As noted by @FlorianASchroeder , PR #250 allowed the union of intraday data only for single-property configurations.
This update removes that limitation by leveraging the fact that both daily and intraday event tables start with 'events_'. This means we can use a wildcard operation to query
events_*to pull both daily and intraday records. Duplicate records will be deduped as part of thebase_ga4__eventsqualify statement.As a result, we can remove the intraday source from
src_ga4.ymlChecklist
dbt testandpython -m pytest .to validate existing tests